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Background / Context: 

Description of prior research and its intellectual context. 

To address concerns about low graduation rates and a lack of workforce readiness, states 
and districts have been experimenting with different models of high school reform. One of the 
most popular models has been the Early College High School (ECHS) model, small schools that 
blur the line between high school and college. Since 2002, over 200 ECHSs have been created 
under the auspices of the Early College High School Initiative, which is primarily funded by the 
Bill & Melinda Gates Eoundation. North Carolina has the largest concentration, with over 70 
ECHSs across the state. Given the rapid expansion of this model, it is critical to understand if the 
model works and for whom it works. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The purpose of this study is to rigorously examine the implementation and impact of the 
Early College High School model in North Carolina. This study is the first to utilize a 
longitudinal experimental design to assess the impact of the ECHS model on student outcomes. 
The study has three main goals: 

• Determine the impact of the model on selected student outcomes; 

• Determine the extent to which impacts differ by student characteristics; and 

• Examine the implementation of the model by site and the extent to which variation in 
specific model components is associated with impacts. 

The primary goal of the ECHS model is to increase the number of students who graduate 
from high school and who continue into and succeed in college. Eigure 1 presents a conceptual 
framework showing the Design Principles of the Early College Model and the program’s 
anticipated key intermediate and long-term outcomes. This model forms the basis for the study’s 
questions on implementation and impact. 

—Eigure 1 about here— 

Eor the purpose of this proposal, we will present core impact findings for students enrolled in 
the study from four cohorts of students (Pilot 1, Pilot 2, Cohort 1, and Cohort 2). At the 2010 
SREE conference, we presented findings for 718 9**^ graders (Bernstein, et ah, 2010) from Pilot 
1, Pilot 2 and Cohort 1. Eor the upcoming conference, we will present 10‘*' grade findings for this 
same sample of students as well as 9^*' grade impacts for an additional 1,044 students who were 
in Cohort 2 of the study. The specific research questions addressed in this paper include the 
following: 

1 . What is the impact of the ECHS model on students’ enrollment and success in core 
college preparatory courses in 9**^ and lO'*' grade? 

2. What is the impact of the ECHS model on students’ behavior in 9* and 10^*' grades, 
including attendance and suspensions? 

3. What is the impact of the ECHS model on students’ persistence in school? 

4. What is the impact of the ECHS model on students’ college-going aspirations? 

5. Do any of these impacts vary by whether students are low-income, the first in their family 
to go to college, or members of a racial/ethnical group underrepresented in college? 
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Setting: 

Description of the research location. 

Primarily located on the eampuses of two- or four- year colleges and universities, ECHSs 
are expected to provide an academically rigorous course of study with the goal of ensuring that 
all students graduate with a high school diploma and two years of university transfer eredit or an 
assoeiate’s degree. The schools in this study are loeated in districts throughout the state of North 
Carolina, in rural and urban settings, and with different student demographics. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features or characteristics. 

Nineteen ECHS in North Carolina are participating in the study. To be eligible for the study, 
sehools had to have more applieants than they had slots and all had to agree to use a lottery to 
select students. Over the course of the study, we have eontinued to recruit schools and students 
through the 2010-2011 sehool year. Most of these sehools agreed to provide multiple cohorts of 
students; for example, in one school, we will have four cohorts of graders. Through the 2010- 
2011 school year, our sample will include an estimated total of 3,100 students in 33 cohorts in 19 
sites. 

Two schools had implemented lotteries prior to the study beginning. Pilot 1 used a lottery to 
deeide who would be aceepted among spring 2005 applicants, and Pilot 2 used a lottery to decide 
who would be accepted among spring 2006 students. After the study began, the research team 
conducted lotteries for Cohort 1 (spring 2007 applicants) and Cohort 2 (spring 2008 applicants). 

The sample for the analyses reported in this paper includes a total of 676 treatment and 
control students in 10^*' grade^ and an estimated 1,700 students in 9^*' grade. Eor eaeh sample, we 
examine baseline characteristics of the treatment and control students to verify that there is a 
statistical balance between the two groups as predicted by the random assignment. Table 1 shows 
the S**’ grade demographie characteristics of the 10* grade sample (Pilot 1, Pilot 2, and Cohort 1). 
As seen, the treatment and eomparison group appear to be statistieally eomparable, except for 
three eharacteristies (retained in the past, and passing math and reading in the eighth grade). A 
similar table will be provided for the expanded 9* grade sample (that includes Cohort 2 students) 
in the final presentation. 

Eor most analyses, students who are missing from the data are excluded from the analyses. 

An exeeption applies to the continuous enrollment outcome, which is explained in more depth in 
the outcomes section. 

— Table 1 about here — 



Intervention / Program / Practice: 

Description of the intervention, program or practice, including details of administration and duration. 

Small schools serving students in grades 9-12 or 13, Early College High Schools are 
designed to increase the number of students graduating from high school and better prepare those 
students for eollege and eareer. At the end of their four or five years of sehooling, students are 
expected to graduate with a high sehool diploma and an associate’s degree or two years of 
transferable college eredit. In order for students to be able to aceomplish this goal of substantial 
credit and a high school diploma, the ECHS must develop, in eollaboration with their higher 



^ In general, students who are not enrolled in North Carolina public schools are excluded from all analyses, with the 
exception of the continuous enrollment outcome. 
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education partner, an aligned, seamless curriculum plan that provides the high school and college 
courses students need to take to complete both degrees and that avoids unnecessary duplication 
and/or omission of critical content. 

Each ECHS is expected to implement and exhibit a specific set of principles, known as 
Design Principles, developed by the NCNSP. Those Design Principles, as articulated by the 
NCNSP and as shown in Eigure 1, are as follows: 

• Ensuring that students are ready for college; 

• Instilling powerful teaching and learning in schools; 

• Providing high student/staff personalization; 

• Redefining professionalism; and 

• Implementing a purposeful design (North Carolina New Schools Project, December, 2007). 

Research Design: 

Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic 
essay, randomized field trial). 

This study is based on a multi-site randomized field trial, supplemented by survey, interview 
and observational data collected on implementation. Prom a pool of eligible students, ECHSs 
enrolled students based on random assignment, and the study compares the students assigned to 
the treatment group (ECHS) with students assigned to the control group (traditional school). 

Each ECHS and the traditional high schools that enroll control group are considered a “site.” 
Therefore, within the pool of applicants to each site, students are randomly assigned to attend an 
ECHS school or to “business as usual,” generally the traditional high school in the district. As 
schools continued to add new 9* grade classes via random assignment each year, those students 
were added to the study sample; hence some sites have multiple cohorts of students. 

With the exception of two pilot schools (their students are in Pilot 1 and Pilot 2) that 
conducted lotteries prior to the study beginning, all lotteries were conducted by the study team. 
Each student was assigned a random number and the list of students was ordered from lowest to 
highest. Schools admitted students in the order in which they appeared on the list. If requested by 
the school, the lottery was stratified to accommodate different needs. Por example, one district 
required that there be proportional representation from each home high school attendance zone. 
Por all students participating in the lottery, the odds of getting in were recorded and used to 
create student-level weights for the impact analyses. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

The data used in the analyses come from administrative data, collected by the North Carolina 
Department of Public Instruction (NCDPI), and merged and de-identified by the North Carolina 
Education Research Center (NCERDC) at Duke University. The specific outcomes examined 
include the following: 

• College Preparatory Coursetaking and Success. We examine outcomes for core college 
preparatory courses, including: English I, Algebra I, Geometry, Algebra II, Biology, and 
Civics and Economics. Por each outcome, we present three measures. The first is whether the 
student took the course or not and serves as a measure of access. The second outcome is a 
traditional pass rate, the number of students who passed the test out of the number who took 
it. The final outcome, and the one we see as most important, is entitled success and collapses 
the first two measures. Success represents the number of students who took the course and 
passed the test. We report this outcome because more students are taking specific courses in 
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the treatment group than in the control group, making straight comparisons of pass rates (i.e., 
of course takers only) problematic. 

• Attendance: Each school reports the number of days each student is present in school, broken 
out by whether the absence was excused or unexcused. 

• Student behavior. Schools report students who have received short term suspensions and long 
term suspensions. 

• Dropout status: When students drop out of school, they are expected to complete a form 
confirming they have dropped out. All students who are identified as having dropped out are 
coded as such by the school, and that information is submitted to NCDPI. These data are 
being cleaned and will be analyzed for the final presentation. 

• Continued enrollment in a North Carolina high school: Because we have concerns about the 
completeness of the dropout dataset,^ we also look at continued enrollment in a North 
Carolina public school as an outcome. This outcome looks at the proportion of students from 
the original 9**^ grade sample still enrolled in a North Carolina school, and is the only 
outcome using the original sample size (including students missing in 10* grade) as a 
denominator, while the others exclude missing students. 

• Aspirations to attend a 4-year college or university. In conjunction with each state-mandated 
test, students are asked to identify their future plans. This outcome is based on the proportion 
of students who indicated they intended to attend a 4-year college or university. 

The impacts of ECHS on these outcomes are estimated within an Intent-To-Treat (ITT) 
framework, in which a student’s initial experimental status as a treatment or control student, 
rather than actual participation in an ECHS, serves as our measure of treatment. We report 
unadjusted means for the two groups. We also report adjusted impacts using logistic regression 
to analyze binary outcomes (e.g., coursetaking, dropouts, etc.) and linear regression for 
continuous outcomes, such as days absent. In our regression-based models, we employ baseline 
student characteristics and site-level indicators (or site fixed effects) as covariates. 

Findings / Results: 

Description of the main findings with specific details. 

Analysis of the 10* grade results shows that the patterns established in 9* grade and 
reported last year (Bernstein, et ah, 2010), continue on through 10* grade. Table 2 shows the 
results from the sample of 9* grade students for whom we are reporting 10* grade results. The 
9* grade analysis will be expanded to include results from an additional 1,000 students from the 
second study cohort. Table 3 includes results for core 10* grade outcomes for treatment and 
control students. 

Impact on students ’ enrollment and success in core college preparatory courses in 
and 10‘^ grade: Eor all core college preparatory subjects, the proportion of ECHS students 
taking and progressing in the courses was higher than the proportion of the students in the 
control group, although not all differences were statistically significant. The statistically 
significant differences occurred primarily in math and science courses. In 9* grade statistically 
significantly more students enrolled and succeeded in at least one core college preparatory math 
course. In 10* grade, statistically significantly more students had taken Geometry (the impact on 



* Because the dropout data are collected separately from all other student data, only 70% of the students in the 
dropout dataset can be linked to other data sets. In addition, there are many students who are not recorded 
formally as dropouts but simply stop coming to school. Thus, not all students who drop out are accurately 
identified as such. 
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success rates was not statistically significant) and statistically significantly more students had 
both taken and succeeded in Algebra II and Biology. The pass rates for these eourses were lower 
in the ECHS, although not statistically significantly so. The lower pass rates in the ECHS are not 
surprising given the much larger proportion of students taking the eourses in the treatment group. 

—Tables 2 and 3 about here— 

Impact of the ECHS model on students ’ behavior in 9‘^ and 10‘^ grades, including 
attendance and suspensions: In both and 10* grade, ECHS students had fewer unexcused 
absences, and statistically significantly lower suspension rates. There was no statistically 
significant difference for either grade in the number of excused absences. It should be noted that 
the 10* grade suspension and attendanee data are currently based on a much smaller sample that 
only includes students from the first two pilot schools conducting lotteries prior to the beginning 
of the study. The final presentation will inelude results for the full sample of 10* graders, as well 
as results for the expanded sample of 9* graders. 

Impact of the ECHS model on students’ persistence in school: Results show that the 
proportion of students still enrolled in North Carolina public schools in 10* grade is statistically 
significantly higher in the ECHS than in the control group (adjusted impact of 4.2 percentage 
points, p=.0007). The dropout data are still being validated and will be incorporated into the final 
presentation. 

Impact on students ’ college-going aspirations: There was no statistically significant 
difference in 9* grade students’ aspirations to attend eollege; however, there was a statistieally 
significant difference in 10* grade students’ aspirations to attend college (adjusted impact of 
10.7 percentage points, p=0.013*). 

Impact by sub-group: Almost all of the impacts that were statistically significant above 
were also statistieally significant when analyzed by subgroup. Table 4 shows whether the 
estimated impacts for a subgroup of students is statistieally significantly different from the 
impaet on the rest of the sample (e.g. the impact on students who are eligible for free/redueed 
price lunch vs. the impaet on those that are not). These results show that the model generally had 
a higher (although not neeessarily statistically significantly higher) impaet on students who are 
members of target populations, particularly minority and low-income students. The statistically 
significantly higher impacts occurred primarily for low-income students in math eourses. 

—Table 4 about here— 



Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

Early results from this study show that the Early College High Sehool model is having a 
positive impact on many outcomes assoeiated with remaining in sehool and becoming ready for 
college. These results suggest that the ECHS is making substantial progress towards its goal of 
graduating more students who are ready for college and work. Although these findings are very 
positive, they are restricted to the model as implemented in North Carolina. Other early eolleges 
around the eountry may not follow the same design principles and may not reeeive the same 
level of assistanee in implementation as the schools in North Carolina; as a result, their results 
may differ. 
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Appendix B, Tables and Figures 

Figure 1: Conceptual Framework of North Carolina’s Learn and Earn Early College High 
School Model 
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Table 1: Descriptive Statistics of the Sample for Cohort 1, Pilot 1 and 2 





Whole 

Sample 

(N=718) 


ECHS Group 
(N=416) 


Control 

Group 

(N=302) 


T-C Difference 




N 


Mean 


N 


Mean 


N 


Mean 


Difference 


P- Value 


Race & Ethnicity 


















American Indian 


718 


0.56% 


416 


0.96% 


302 


0.00% 


0.96% 


0.088 


Asian 


718 


1.25% 


416 


1.44% 


302 


0.99% 


0.45% 


0.594 


Black 


718 


21.45% 


416 


21.63% 


302 


21.19% 


0.44% 


0.887 


Hispanic 


718 


5.57% 


416 


5.77% 


302 


5.30% 


0.47% 


0.786 


Multi racial 


718 


2.92% 


416 


2.40% 


302 


3.64% 


-1.24% 


0.332 


White 


718 


68.25% 


416 


67.79% 


302 


68.87% 


-1.09% 


0.758 


Gender 


















Male 


717 


38.49% 


415 


38.07% 


302 


39.07% 


-1.00% 


0.786 


Age 


635 


15.35 


371 


15.32 


264 


15.39 


-0.07 


0.086 


Exceptionality 


















Disabled/Impaired 


687 


3.78% 


409 


3.67% 


278 


3.96% 


-0.29% 


0.846 


Gifted 


701 


11.98% 


410 


11.95% 


291 


12.03% 


-0.08% 


0.976 


First Generation 
College 


703 


45.80% 


406 


43.84% 


297 


48.48% 


-4.64% 


0.223 


Free/Reduced Price 
Lunch 


705 


44.40% 


405 


43.95% 


300 


45.00% 


-1.05% 


0.782 


Retained 


647 


2.47% 


377 


0.80% 


270 


4.81% 


-4.02% 


0.001* 


8"’ Grade 
Achievement 


















Math - pass 


691 


81.91% 


401 


84.79% 


290 


77.93% 


6.86% 


0.021* 


Reading - pass 


689 


97.82% 


402 


98.76% 


287 


96.52% 


2.24% 


0.047* 


Algebra 1 - pass 


182 


97.25% 


115 


96.52% 


67 


98.51% 


-1.99% 


0.432 



Notes: Statistically significant differences (at the p<0.05 level) are denoted by *. 

This table will be updated with data from the Cohort 2 sample containing an additional 1,000 students. 
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Table 2: 9*'’ Grade Outcomes: Unadjusted Group Means and Adjusted Impact Estimates 
for Cohort 1, and Pilot 1 and 2. 





Unadjusted Means 
(N=413) 


Adjusted Impact 
(N=294) 


ECHS 


Control 


Estimate 


P-Value 


Algebra I 


% Take-up 


97.0 


76.3 


11.3 


<0.001 


% Pass (takers only) 


83.4 


88.1 


-4.0 


0.081 


% Progress 


80.9 


67.3 


6.4 


0.080 


English I 


% Take-up 


95.8 


92.9 


1.4 


0.190 


% Pass (takers) 


91.3 


89.6 


0.2 


0.900 


% Progress 


87.4 


83.3 


1.3 


0.569 


College Prep. Math Course-taking 


% At least one eourse take-up 


97.5 


76.3 


11.0 


<0.001 


% At least two eourses take-up 


38.6 


22.7 


10.2 


0.005 


% At least one eourse progress 


82.6 


67.3 


7.9 


0.030 


% At least two eourses progress 


34.1 


22.5 


7.0 


0.137 


Attitudinal and Behavioral Outcomes 


Excused Absences (days) 


2.9 


2.9 


-0.2 


0.556 


Unexcused Absences (days) 


3.8 


6.4 


-1.5 


0.029 


% Suspended at least once 


2.7 


20.6 


-12.9 


<0.001 


% Planning to attend 4 yr college 


73.2 


68.7 


2.1 


0.59 



Note: This table will be updated with data from the Cohort 2 sample containing an additional 1,000 students. 
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Table 3: 10*'’ Grade Outcomes: Unadjusted Group Means and Adjusted Impact Estimates 
for Cohort 1, and Pilot 1 and 2 . 



Outcomes 


Unadjusted Means 


Adjusted Impact 


ECHS 

(N=399) 


Control 

(N=277) 


Estimate 


P-Value 


Algebra I 


% Take-up 


98.7 


90 


2.9 


0.008* 


% Pass (takers) 


84.3 


84.4 


-2.3 


0.357 


% Progress 


83.3 


76 


0 


0.995 


Geometry 


% Take-up 


84.4 


67.1 


16.2 


<0.001* 


% Pass (takers) 


84.3 


88.5 


-4.6 


0.057 


% Progress 


71.1 


59.4 


7.2 


0.128 


Algebra II 


% Take-up 


56 


31 


22.2 


<0.001* 


% Pass (takers) 


81.2 


89.6 


-5.8 


0.178 


% Progress 


45.4 


27.8 


9.9 


0.009* 


Civics and 
Economics 


% Take-up 


91.6 


83.1 


2 


0.207 


% Pass (takers) 


87.1 


85.1 


1.6 


.498 


% Progress 


79.8 


70.8 


3.5 


0.292 


English I 


% Take-up 


98.7 


96 


0.6 


0.101 


% Pass (takers) 


92 


89.4 


0.8 


0.635 


% Progress 


90.8 


85.8 


1.6 


0.412 


Biology 


% Take-up 


76.8 


57.9 


16.3 


<0.001* 


% Pass (takers) 


87.9 


91.4 


-3.3 


0.077 


%Progress 


67.5 


52.9 


10 


0.033* 


College Prep. 
Math Courses 


%At least one course take-up 


99 


90 


2.9 


0.014* 


%At least two courses take-up 


92 


67.8 


18 


<0.001* 


% At least three courses take- 
up 


48.1 


30.4 


14.7 


0.002* 


% At least one course progress 


88.2 


77.6 


4.3 


0.070 


% At least two courses 
progress 


72.9 


57.9 


7.3 


0.109 


% At least three courses 
progress 


38.7 


27.8 


5.3 


0.205 


Additional 

Outcomes 


Continued Enrollment (%) 


94 


89 


4.2 


0.007* 


Excused Absences (days)' 


2.9 


3.1 


-0.6 


0.379 


Unexcused Absences (days)' 


4.6 


8.1 


-3 


0.074 


Suspensions (% suspended at 
least once)* 


5.5 


20.9 


-9.6 


0.006* 


% Planning to Attend 4 Yr 
College 


75.8 


61.9 


10.7 


0.013* 



Notes: Statistically significant differences (at the p<0.05 level) are denoted by *. 

'Absences and suspension data only include pilot 1 and pilot 2; they will include Cohort 1 students in the final 
presentation. 
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Table 4: 10**' Grade Outcomes: Comparison of Subgroup Impacts 





Outcomes 


Minority 
(N=I89) vs. 
Non-minority 
(N=487) 


Firstgen 
(N=296) vs. 
Non-Fir stgen 
(N=368) 


Free i 
(N=2’: 
Non- 
Funch ( 


_,unch 
^5) vs. 
Free 
;N=385) 






Diff 


P-Val 


Diff 


P-Val 


Diff 


P-Val 


Algebra I 


% Take-up 


4.3 


0.246 


4.5 


0.063 


5.4 


0.12 




% Pass (takers) 


-6.2 


0.247 


-6 


0.276 


-6.5 


0.293 




% Progress 


-6.5 


0.324 


-3.7 


0.572 


-6.8 


0.355 


Geometry 


% Take-up 


12.9 


0.129 


6.2 


0.431 


16.9 


0.056 




% Pass (takers) 


2.1 


0.747 


-2 


0.707 


6.4 


0.32 




% Progress 


16.6 


0.111 


3.4 


0.719 


9.2 


0.353 


Algebra II 


% Take-up 


8.9 


0.311 


-1.5 


0.872 


18 


0.056 




% Pass (takers) 


4.8 


0.318 


-6.8 


0.411 


12.5 


0.129 




% Progress 


4.1 


0.588 


-1.2 


0.876 


13 


0.109 


Civics and 


% Take-up 


3.1 


0.519 


9.7 


0.019 


11.3 


0.007* 


Economics 


% Pass (takers) 


2.7 


0.723 


-3.7 


0.522 


6.3 


0.327 




% Progress 


5.5 


0.525 


1.3 


0.86 


17.7 


0.017* 


English I 


% Take-up 


-0.7 


0.332 


1.9 


0.115 


0.4 


0.346 




% Pass (takers) 


2 


0.655 


1.5 


0.698 


-2.5 


0.502 




% Progress 


1.2 


0.811 


2.8 


0.515 


-0.6 


0.886 


Biology 


% Take-up 


13.7 


0.132 


11.9 


0.16 


22.3 


0.015* 




% Pass (takers) 


-0.5 


0.93 


1.3 


0.721 


3.1 


0.584 




%Progress 


14.6 


0.172 


10.7 


0.25 


23.1 


0.015* 


College Prep. 


%At least one course take-up 


5.1 


0.165 


5.6 


0.008* 


1.6 


0.448 


Math Courses 


%At least two courses take-up 


6.7 


0.406 


11.4 


0.089 


20.2 


0.004* 




% At least three courses take- 
up 


14.9 


0.087 


-2.4 


0.792 


15.7 


0.081 




% At least one course progress 


5.5 


0.37 


2.3 


0.674 


4.2 


0.481 




% At least two courses 
progress 


12.7 


0.266 


0.5 


0.957 


5.2 


0.594 




% At least three courses 
progress 


7.4 


0.387 


-0.7 


0.932 


13.4 


0.099 


Additional 


Continued Enrollment (%) 


-2.7 


0.39 


6.1 


0.09 


0.2 


0.951 


Outcomes 


Excused Absences (days)' 


1.4 


0.252 


0.8 


0.62 


2.9 


0.042* 




Unexcused Absences (days)' 


0.5 


0.831 


1.8 


0.566 


-5 


0.477 




Suspensions (% suspended at 
least once)* 


-4.6 


0.576 


3.4 


0.643 


-7.4 


0.36 




% Planning to Attend 4 Yr 
College 


7.3 


0.473 


-24.4 


0.005* 


4.8 


0.602 



Notes: Statistically significant differences (at the p<0.05 level) are denoted by *. 
'Only includes pilot 1 and pilot 2 
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